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Abstract: The contemporary development of education is marked by the rapid expansion of online educational applications 
and platforms. Consequently, it is becoming clear that teachers must consider them and in light of their changing role, know 
what they can offer in their field to be able to recommend them to students to get additional knowledge or even to integrate 
them into their teaching. As students in the field of computer science and informatics are particularly accustomed to considering 
online knowledge resources, we decided to investigate their perception of educational applications and platforms. Based on 
the analysis of the results of their testing of educational platforms and applications, we found that for them the most important 
is content, followed by personal preferences, reason, user experience, price, etc., and only in the last place is a certificate. The 
most frequent word in their research reports proved to be knowledge, which we included under the code reason, followed by 
research, content, time, free, variety, and quality. It also turned out that students’ experience of testing educational applications 
and platforms is predominantly positive and has even improved over the course of the last three years, which we attribute to 
the effects of the Covid-19 coronavirus epidemic. The comparison of pre-pandemic and post-pandemic data also revealed that 
positive sentiment came to the front, while students now prioritize user experience, reason, and quality over the content and 
personal preferences compared to the pre-pandemic period, while they are still aware of the need for exploration. 
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Introduction 


Teachers play many roles in classrooms, including inspiring students, stimulating their interest in 
the subject, and serving as a source of knowledge and information. Besides, commitment and passion are 
required, while a combination of teaching paradigms is needed to maintain momentum and transform the 
classroom into a learning environment full of enthusiasm and interest (Powell et al., 2012). 

Teachers increasingly complain that their students do not show interest, do not prepare for lessons, 
have passive attitudes toward learning activities, and limited awareness of their own learning process 
(Buijs and Admiraal, 2013). This, and the contemporary competence-oriented higher education, requires 
a correspondingly strong focus on pedagogy. But there is also a growing recognition that to develop 
the knowledge and skills needed for the 21* century, we also need innovations in the field of teaching 
(Paniagua and Istance, 2018). 

As explained by Heacox and Pengal (2009), the greatest challenge for today’s teacher is to respond 
to the increasingly diverse learning needs, styles, and social backgrounds of students, which affect their 
willingness to learn, learning styles, motivation, interests, attitudes to learning, and self-confidence. He, 
therefore, suggests that differentiation of instruction, which puts students at the center of teaching and 
learning and also allows learning needs, learning styles, and interests to guide instructional design, can 
be very helpful, including varying the pace of learning, the level of difficulty and the way instruction is 
delivered (Heacox and Pengal, 2009). 

The traditional lecture is not an effective learning environment for many students, as many do 
not actively participate during a traditional lecture (Bajpai, Biberman and Ye, 2019). The goal is to keep 
students active, and approaches to learning are changing. In student-centered learning, the focus is on 
students’ needs, abilities, interests, and learning styles, and the teacher is only a facilitator of learning. Due 
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to the development of the World Wide Web and search engines, ‘on-demand’ information has become a 
reality (Hirsh, 2018). In this context, students need to be active and responsible participants in the learning 
process, and the teacher plays a key role in the whole process. 

E-learning uses various information-communication technology (ICT) tools to make the teaching 
and learning process more effective (Bajpai, Biberman and Ye, 2019). As a result, classroom lectures, 
whose main objective is to cover the content of the course, can be expected to become obsolete, as can 
the willingness of students to attend these classroom lectures, as technology allows them to learn at 
their own pace and in their own time (Moreno, 2018), expanding the possibilities for individualisation and 
personalisation of learning (Urbancic, 2021). More and more people are realising the benefits of learning 
at a pace that suits them best, which is why e-learning is on the rise. 

E-learning allows the teacher and the student to be independentin terms of location and time, making 
it a form of distance learning (Bregar, Zagmajster and Radovan, 2010). Furthermore, the advantage of 
online education is that many courses are very affordable for the value they offer and are aimed at almost 
every individual in the world, whereas traditional education involves a limited number of students (Krastev, 
2019), which is an additional reason for the rapid rise of educational apps and platforms. 

While the concept of e-learning generally refers to the methods of education implemented using 
ICT, ICT in the context of e-learning represents only part of the technological solutions that allow us to 
complement or enrich the basic foundations and concepts of traditionally based learning (Jagodié, 2010). 
Nevertheless, ICT is a potentially powerful tool for expanding educational opportunities, both formal and 
non-formal (Adenusi, Adebayo and Oni, 2019), and it is the development of ICT that has brought about 
significant changes in the field of education (Semerci and Aydin, 2018). 

However, we recognise that teachers need to have certain digital competencies to be able to use 
online platforms and applications for communication, teaching, and research. Therefore, national projects 
to promote the use of ICT by (higher education) teachers are very welcome. Teachers play a central role in 
integrating ICT into the classroom, so improving their digital competencies and attitudes is crucial for the 
effective integration of ICT in education (Semerci and Aydin, 2018). In addition, it is important to be aware 
that today’s students are so-called digital natives, who are expected to have the knowledge and skills that 
allow them to handle ICT tools ‘naturally’ (Boh Podgornik et al., 2016), so it is not surprising that students 
often expect this from teachers as well. 

The introduction of the use, integration, and dissemination of ICT has ushered in a new era in 
educational methodologies, radically changing traditional teaching methods and learning patterns in the 
field and offering modern learning experiences for both teachers and students (Adenusi, Adebayo and Oni, 
2019). Therefore, different teaching/learning strategies need to be applied in education. This is because 
digital technologies are often used to support the extension of university education into the home, social 
and work environments (Castafieda and Selwyn, 2018). 

As a result, ICT has a broader role in teaching and learning activities into which it can be integrated 
as a tool in the learning process (Lautenbach, 2014). Furthermore, the important role of ICT can also 
be found in activities such as content creation, student performance monitoring, training, knowledge 
management and organisation, and formative monitoring (Margaret et al., 2018). Black and Wiliam 
(2009), two of the most referenced authors, established a conceptual framework for understanding 
formative monitoring through a series of publications that can be traced back at least to the mid-1990s. 
Formative monitoring can be imagined as a “bridge between learning and teaching”. It is based on five key 
strategies: clarification, participation in setting and understanding learning intentions and success criteria; 
designing classroom activities that provide evidence of learning; providing feedback; activating students 
to become teachers to each other; and activating students to self-manage their learning (Palir Mavric, 
2017). The integration of ICT was identified as a good practice, particularly in stimulating and increasing 
students’ motivation. But the use of technology in learning does not automatically result in engagement 
in learning activities and improved learning outcomes (Sinha et al., 2015), and for this reason, well- 
designed computer-supported collaborative learning (CSCL) environments for collaborative learning and 
problemsolving might intensify learning if used based on a teacher’s instructional support (Pietarinen, 
Palonen and Vauras, 2021). 

Moreover, teachers and students must understand that independent learning using online 
applications and platforms has a major disadvantage if not carefully addressed and that is a lack of 
collaboration. Namely, students generally do not recognise the advantages of collaborative learning, and 
therefore teachers should play an important role in encouraging and influencing student participation and 
achievement (Webb et al., 2019). The teacher’s role has thus changed to act as a coach and facilitator of 
students’ thinking while modelling the learning process (Anderson, 2002). 

ICT tools are thus playing an increasingly important role in the renewal of educational methods. 
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This is manifested in the form of digital learning materials instead of printed books, interactive materials, 
e-communication between teachers and students, etc. (Berényi and Deutsch, 2018). While the pandemic 
has increased the use of and changed teachers’ attitudes toward digital learning materials, quality teaching 
in the digital age would require supporting teachers with training to make working with digital materials 
routine (Mohar and Kovaé, 2021). ICT tools used to create and disseminate knowledge including equipment 
and software solutions have become an indispensable part of modern culture, which is spreading globally 
through education (Bajpai, Biberman and Ye, 2019). Software that supports the didactic process in higher 
education includes internet browsers, office suites, communicators, specialised software suitable for 
classroom use, software that supports group work, and above all e-learning applications and platforms 
(Grooms, 2018), which are the focus of our research. 

As the use of educational applications and online platforms is growing rapidly (e.g., Data Bridge 
Market Research, 2022 or Dhawal, 2021), we decided to explore their use for study purposes. To this end, 
we analysed the opinions of students from four Informatics and Computer Science degree programmes, 
two first-degree programmes and two second-degree programmes. 

We started the survey in 2019 when we first asked students enrolled in their final year of study on 
these four study programmes about which apps and online platforms, they use to support their studies. 
Based on the responses we received, we found that the use of apps and online platforms is much more 
widespread among students and not limited to those suggested by teachers. In addition, the students 
presented in their research reports the results of the testing of the apps and online platforms they use to 
complement their studies. 

As today’s teachers (and especially teachers in information technology) and computer science 
programmes, due to the specificity of these programmes) are increasingly aware of the need to use 
and even compete with online educational opportunities, it has become clear that they need to change 
their teaching methods and see their profession as adding value and upgrading the information that 
students can obtain via the Internet. Our main research objective is therefore to present the findings of a 
qualitative analysis of students’ views when testing educational applications and online platforms, which 
we believe could encourage higher education teachers to make greater use of them in their teaching, 
not as a substitute, but as an additional source of ‘on-demand’ knowledge and motivation for learning. 
Finally, this paper aims to encourage higher education institutions to reflect on how encouraging the use 
of educational apps and online platforms can enable students to perform even better in exams, while also 
spreading awareness that they can deepen their knowledge in this way too. 


Materials and Methods 


Statistics (e.g., Wise, 2022) show that e-learning has grown exponentially in recent years. Studies 
also show (e.g., Suwal and Singh, 2018) that the use of e-learning applications and platforms is well 
accepted by students as it gives them positive experiences, and consequently, there is a growing need 
for more integration of e-learning applications and platforms into existing learning activities and practices. 
In the flood of different types of educational apps and online platforms, it is difficult for a teacher to decide 
which app or platform is appropriate to use and recommend to students. In addition, more and more 
new apps and online platforms are emerging, making it difficult to keep up with these advances and the 
increasing range. In this context, we have identified the need for a systematic review of the educational 
apps and platforms that students individually choose to use, as well as the way they are perceived. To do 
so, we have conducted a survey in which students themselves, based on their study interests, chose an 
app or online platform to test. They presented their views gained during the testing in research reports, 
which we then used to achieve the aim of our research: to show that teachers need to change their role, 
as the internet offers instant information on almost any topic and as educational apps and platforms are 
gaining in importance. Accordingly, teachers also need to change their teaching methods to offer students 
all they can gain from e-learning. 

Based on the content of the students’ research reports, we identified the codes and the most 
frequently written words to give their perspective on the impact and usefulness of some educational apps 
and platforms in their fields of study, as the research also aims to encourage teachers to reflect on how 
the integration of apps and platforms in their courses can bring added value. In addition, the Covid-19 
coronavirus pandemic has further stimulated the development of online educational resources. Based on 
this research objective, the following research questions (RQs) were set: 

RQ1: What has proved to be most important in the testing of educational apps and platforms by 
students? 
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For this question, we were looking for codes that could be extracted from a close reading of the 
assignments that students had prepared based on testing educational apps and platforms. We assumed 
that students would put the user experience at the center, and quality would be the decisive factor (as in 
Rojko, 2020). On the other hand, although some apps and platforms provide certificates and recognition 
of the knowledge acquired, we assumed that the absence of formal recognition would demotivate students 
to learn online on their own initiative. 

RQ2: Which words stand out in the research reports of students who have tested educational apps 
and platforms? 

We asked students to use different criteria to test different applications and platforms for their 
research reports, mainly qualitative but sometimes also quantitative. We anticipated that the words 
quality, user experience, quantity, price and certificate and their synonyms would appear most frequently. 
We were also interested in whether positive or negative experiences predominated. 

RQ3: Has the impact of the Covid-19 coronavirus pandemic changed the way students view 
educational apps and platforms? 

We expected that some new dominant words would appear in the more recent research reports 
(from the second half of the academic year 2021/22) compared to the reports from the period before the 
start of the epidemic (from the first half of the academic year 2019/20). Namely, we observed that the 
Covid-19 pandemic was reflected in an increased motivation of students to search for additional online 
education resources and in their increased willingness to try them out, as distance education had been 
the only option for some time. 

The survey was launched in 2019 when we asked students enrolled in the final year of four 
Informatics and Computer Science degree programmes from Slovenia, which educational apps and 
platforms they use to support their studies. In addition, the majority of these students also produced a 
research seminar paper based on the results of their testing. 

Students chose and tested the following educational apps and/or platforms (in alphabetical order): 
ArtofProblemSolving, Astra.si, Bussuu, CK-12, Code With Mosh, CodeAcademy, Codewars, Coursera, 
Cybrary, Drops, Duolingo, edX, freeCodeCamp, FreeVideoLectures, FunEasyLearn, Khan-academy, 
KodeKloud, Laracast, LinkedIn learning, Mathway, Memrise, Micro:bit, Mimo, MojeZnanje.si, Photomath, 
Pluralsight, Programming Hub, Qlango, Scratch, Skillshare, SoloLearn, SQLZOO, Stack Overflow, 
Symbolab, Treehouse, Tutorialspoint, Udemy, Udacity, W3Schools, Wolfram Alpha, Yoast and YouTube. 
Note that some students chose the same apps and platforms, and Udemy was the most repeated platform, 
18 times. 

Some of the educational apps and platforms chosen specialise in one area, others cover several 
different areas, and students were instructed to focus on one specific subject area for any comparisons. 
As can be seen from the list of names, some of the listed platforms cover different areas of knowledge, 
although all students focused on computer science, while the listed apps focus on computer science (most 
frequently tested), mathematics (medium frequency), and foreign language learning (least frequently 
tested). The majority of the apps and platforms analysed were international, but two Slovenian platforms, 
Astra.si and MojeZnanje.si, were also included. 

Among the 140 students enrolled in the final year of their first (professional study programmes) 
and second (masters study programmes) cycle studies in the academic years 2019/20, 2020/21 and 
2021/22, 113 (81%) submitted their research seminar paper. Among these, we further excluded those 
reports in which students tested other applications that they had used during their studies but were not 
relevant to our research - e.g., for storing data and literature, such as Dropbox, Evernote, Google Drive, 
OneNote and Zotero. Thus, the final 90 research reports (50 reports by first-cycle students and 40 reports 
by second-cycle students) were considered, out of these 29 from the academic year 2019/20 (first half 
of the year (in continuation referred to as the pre-pandemic period) 16 reports), 36 from 2020/21, and 25 
from 2021/22 (second half of the year (in continuation referred to as post-pandemic period) 10 reports). 
We considered the content of research reports as transcripts, which were then subjected to qualitative 
analysis to answer our research questions. We resorted to manual coding and used the Atlas.ti qualitative 
analysis tool. 

Manual coding (using the open axial coding methodology) required close reading and careful 
consideration of the students’ opinions and observations, taking special care to avoid concepts we had 
learned from studying the literature and our views and expectations. The attribution of concepts was 
done both by direct conceptualisation and synonymy. The selection was made based on the conceptual 
framework and within it based on recurring terms in the students’ assignments. Individual terms were 
further evaluated and linked to semantically equivalent terms, which were grouped into key categories 
(codes) based on their meaning. 
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Atlas.ti was used to search for the most frequently repeated words in the assignments. Although this 
tool also allows the preparation of word clouds based on Slovene texts, it only formats them appropriately 
in some of the world’s leading languages. As the tool is not able to combine Slovenian words in different 
conjugations, does not correctly identify word types, etc., we first translated the 90 analysed seminar 
reports, with an average size of one author field (The author’s pole consists of 30 000 characters (letters, 
numbers, punctuation, spaces) or approximately 16 typed pages.), into English. We then extracted nouns, 
verbs, adjectives, and adverbs, followed by manual removal of the remaining inappropriate words. Finally, 
for the most common words, we first considered separately the terms for the full spectrum of experience 
from positive to negative, and then all the remaining most commonly used words. 

In addition, we also analysed the research reports from the first half of the 2019/20 academic 
year (considered as pre-pandemic period reports) and the research reports from the second half of the 
2021/22 academic year (considered as post-pandemic period reports) separately and used the results 
to compare the two periods, which led us to detect the changes in the testing experience as well as in 
students’ attitudes towards educational apps and platforms. To do so, we applied chi-squared test at 
significance level of 0.05 to find potential association between pre- and post-pandemic periods regarding 
the testing experience and students’ attitudes. In the case of a statistically significant chi-squared test, the 
association was confirmed and was then tested for strength using the Cramer's V contingency coefficient 
at 1 degree of freedom. We used the value of the latter to determine whether the association was weak 
(Cramer’s V value between 0.1 and 0.3), moderate (Cramer’s V value between 0.3 and 0.5) or strong 
(Cramer’s V value between 0.5 and 1.0). To visually represent the observed associations, we used mosaic 
plots in which two-dimensional frequency table is displayed using rectangles. The rectangular regions 
in the mosaic plot are proportional to the cell frequencies they represent, where shadings indicate the 
residuals between observed and expected frequencies. Positive value of a residual is represented by the 
blue color, and negative value of a residual is represented by the red color, whereas the shade of color 
indicates the site of residual: the larger the absolute value of residual, the darker the color, and vice versa, 
the smaller the absolute value of residual, the lighter the color. This is also presented in the legend added 
to the mosaic plot (Kabacoff, 2015). 


Results 


We first present the results of the manual coding (RQ1) and then the results of the analysis using 
the Atlas.ti tool, which was used to produce linear word clouds for all analysed research reports together 
(RQ2) and separately based on research reports from before the Covid-19 epidemic and two and a half 
years later (RQ3). 


Coding results 

Based on a close reading of a selection of research reports (N=90), we identified 10 key content 
categories (codes) in the texts: content, personal preference, reason, user experience, price, quantity, 
basic, development, quality, advanced and endorsement. 


traditional 


content 


development 
personal 
preferences 
personal 
educational apps educational apps ae 
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eee and platforms Meee = and platforms 
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Figure 1. Comparison of paradigm models with Rojko (2020) 
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Figure 1 shows that our analysis produced slightly different results than Rojko (2020) as interaction, 
motivation, recommendation, and traditional were replaced by reason, development, and content, in 
Figure 1 right, according to our findings. Similarly, Rojko (2020) found that quality dominates over user 
experience and quantity, whereas our findings showed a different picture — as presented in the Figure 2. 
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7000 
certificate 6000 personal preferences 
5000 
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advanced 3000 reason 
2000 
1000 


quality user experience 


development price 


basic quantity 


Figure 2. Representation of each dominant word by code 


Figure 2 shows the representation of each of the dominant words by codes (We did not add the code 
area to the selection (dominant words: coding, programming, foreign, language(s), technology) because 
it is not relevant, as these are the fields of study of the students whose research seminar papers are 
analysed.), which were written at least 200 times in all research reports combined. It turns out that content 
stands out the most (words: content, tutorial, example, material, new, video, game, help, information, 
data), followed by personal preferences (words: option, year, years, time, different, decision, way, 
possibility, relevant, important, choose, criterion, need, work). For content, we would highlight the words 
video, game, and help as they best convey the students’ observations, while for personal preferences we 
would highlight time, different, and way as they best convey the students’ thinking. 


Dominant words 

The dominant words were extracted using Atlas.ti tool, but despite the translation of the analysed 
research reports into English, a lot of manual refinement was necessary (see Appendix 1 for a full list of 
removed words). The words that appear in Figure 3 appear at least 200 times in the analysed reports. The 
most frequently occurring word is language (N=1476; also, languages in 16th place (N=657)), which in turn 
refers to the domains of the applications and platforms tested (computer languages, foreign languages). 
This is also the case for the third-ranked word programming (N=1342), the twelfth-ranked word coding, 
etc. The second-ranked word is knowledge (N=1419), which we have classified under the code reason. 
The results based on the most frequent individual words thus show that knowledge is the most important 
for students, research is needed, followed by content, time is a constraint, and free, variety, and quality 
are also among the most important. 

It should also be noted that it is necessary to group certain words that appear separately in the 
figure below, e.g. research (N=1318) and search (N=235), information (N=595) and data (N=508), find in 
the present and past tense (N=559 and N=371), video (N=459) and videos (N=270), year (N=238) and 
years (N=204), play (N=358) and games (N=205), etc. 
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Figure 3. Dominant words appearing at least 200 times (all assignments). 


The sentiment analysis provided by Atlas.ti, without manual cleaning of the results, shows that 
negative sentiment prevails (2305 in paragraphs and 4131 in sentences) over positive sentiment (1756 
in paragraphs and 2593 in sentences). But a quick look at the results shows that a lot of manual cleaning 
would be needed, as the sentiments are very often improperly attributed. Therefore, to identify the 
experience from positive to negative, we also focused on the most frequently written words. This revealed 
that the positive experience of students’ testing educational apps and platforms is the predominant one, 
and Figure 4 shows the words that were used most frequently to describe the experience. These words 
are also included in Figure 3. 


& 


Figure 4. Dominant experience words from positive to negative, occurring at least 200 times (all 
assignments) 


As shown in the figure above, 81% (N=2,781) have a positive sentiment. We would add that the 
word mediocre also appeared 215 times among the most frequent words, but we did not include it in 
Figure 4. If we had evaluated it as a neutral sentiment, the positive sentiment would have dropped to 
76%. The limitation is that the included words are subjectively classified as sentiment and that only words 
that appear at least 200 times in all research reports are used, but the result is clear. Again, in personal 
conversations with each student, we perceived an overwhelmingly positive attitude, although occasionally 
they encountered some problems when taking the test. 
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Comparison 

Following the findings presented above we also carried out a comparative analysis between the 
periods before and after Covid-19 pandemic. In this context, we have referred to the research reports from 
the first half of the 2019/20 academic year as pre-pandemic period research reports, and the research 
reports from the second half of the academic year 2021/22 as post-pandemic period research reports. 

The analysis of the pre-pandemic period research reports considered 16 reports, and the analysis 
of the post-pandemic period research reports considered 10 reports. They both involved a lot of manual 
cleaning and Figure 5 shows the result - words that appear at least 50 times. Considering the pre- 
pandemic period reports, if we remove the words related to the fields of study (programming, coding, and 
language), we see that the most important are content, reason, and personal preferences, and the top 
words within the codes are content (N=352), followed by knowledge (N=275) and research (N=166). Also 
standing out are way, different, video, time, new, experience, find, and information. Considering the post- 
pandemic period research, we notice that the words relating to the fields of study do not appear first in the 
top positions (in fact, only language appears in the list, in the 15" position). 


contentprogramming 


knowledgé research code way different video 


time new language education experience find good information 
help skills number case level need better game available free quality 
development process work languages problem training want example important results 
access data basic result found mathematical suitable many solution videos mobile maths 
solving description model problems several solve well possible much community search 
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language free find easy case model price time search interface important many want well helpful 
mobile quality suitable way video results step useful beginners much covered great level training 


Ss 
post-pandemic period 
reports 


Figure 5. Dominant words appearing at least 50 times: (a) research reports from the first half of the 
academic year 2019/20 (pre-pandemic period reports), and (b) research reports from the second half of 
the academic year 2021/22 (post-pandemic period reports) 


However, in the post-pandemic period reports content is still the most important for students, while 
personal preferences and reason changed their position, as personal preferences came as second most 
important. Namely, the top words within the codes are content (N=311), followed by experience (N=177) and 
knowledge (N=172). Rating, criteria, education, information, data, different, and decision also stand out. 
This suggests that students became more focused on their own preferences when choosing educational 
applications and platforms, likely because they became more acknowledged with the (enlarged) offering 
compared to pre-pandemic. 

The comparison was applied to the sentiment analysis as well. Figure 6 shows the words that 
were used most frequently to describe the experience in pre- and post-pandemic period reports. We 
acknowledge that od the sentiment in pre-pandemic period reports, good comes first, followed by better, 
then problem/issues. Positive sentiment dominates with 64%. In post-pandemic period reports, however, 
sentiment has come to the front, with the words good and better appearing in the top five. 
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pre-pandemic period reports post-pandemic period reports 
Figure 6. Dominant experience words from positive to negative: (a) research reports from the first half of 
the academic year 2019/20 (pre-pandemic period reports), and (b) research reports from the second half 
of the academic year 2021/22 (post-pandemic period reports) 


Thus, in post-pandemic period reports positive sentiment is 100%, as the words problem or 
problems do not appear, which is one of the most important differences compared to the results of the 
students’ test results two and a half years earlier. This is per our belief due to the changed attitude of 
students towards this type of learning, but also to the rapid development of educational applications and 
platforms, which has been accelerated by the pandemic. The word mediocre appears, which, if taken into 
account, could reduce satisfaction with testing by 33%. 

In order to support the findings presented above, we also performed chi-square tests at significance 
level 0.05 where the strength of the association was determined by the value of the Cramer's V contingency 
coefficients. Figure 7 represents the mosaic plots for a) dominant words, and b) dominant experience, in 
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Figure 7. Comparison considering (a) dominant words, and (b) dominant experience in the pre- and post- 
pandemic period reports 


To compare the pre- and post-pandemic period reports regarding the dominant words, we have 
considered only those words that appear among the dominant ones in both periods. In this case we 
observe significant (p < 0.05) but weak (Cramer's V = 0.184) association between pre- and post-pandemic 
periods. The discrepancies between the two compared periods occur in the case of the words different, 
experience, and knowledge, where in the post-pandemic period experience has significantly gained in 
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importance, while the importance of different and knowledge has declined significantly as compared to 
the pre-pandemic period. 

Considering sentiment analysis, we observe strong (Cramer's V = 0.482) and significant (p < 0.05) 
association between dominant experience in pre- and post-pandemic period reports, which is consistent 
with the fact, that positive sentiment among post-pandemic research reports is 100%, as the words 
problem or problems do not appear. This finding can most likely be attributed to the rapid development of 
educational applications and platforms, and the accelerated spread of their use as a consequence of the 
pandemic. 


Discussions 


The methodology of extracting the main findings from the students’ research reports, where we 
paid attention to the students’ opinions and observations without combining them with our views and 
expectations, proved to be a difficult task but helped us to provide neutral answers to our research 
questions. The content analysis of the students’ reports was carried out using manual coding and the 
qualitative analysis tool Atlas.ti provided a detailed insight into the students’ opinions. Below we summarise 
the main findings that answered our research questions: 

RQ1: What has proved to be the most important in the testing of educational apps and platforms 
by students? 

The most important category was content, where we would point out the following most frequently 
written words: video, game (including tutorial, example), and help. This is followed by the category 
of personal preference, as students supplement their education with educational apps and platforms 
according to their possibilities, needs, and time, and variety and method are also important to them. 
This is because nowadays, the diversity of learning needs, styles, and social backgrounds of students 
are increasingly coming to the fore in education (Heacox and Pengal, 2019), which the second-ranked 
personal preference code confirms. Reason, user experience, price, and other categories follow. 

However, we expected that, although some apps and platforms provide certificates of knowledge, 
the absence of official recognition might demotivate students to learn online on their own initiative. But 
we found that formal education is not the only criterion for some companies that also seek recruits via 
e-learning platforms and applications (in the cases studied, these are so far only foreign companies), and 
the qualitative analysis also resulted in the category of certificate appearing as a certificate in the word 
cloud (N=303) being ranked last. 

RQ2: Which words stand out in the research reports of students who have tested educational apps 
and platforms? 

We expected that the words quality, user experience, quantity, price, and certificate, and their 
synonyms (as in Rojko, 2020) would appear most often. But it turned out quite differently. If we exclude 
the most frequent words related to fields of study, the word knowledge ranks highest, which we have 
placed under the category of reason, followed by research, content, time, free, variety, and quality. The 
word certificate is ranked only 52". 

It also showed that students’ experience of testing educational apps and platforms is predominantly 
positive (81%), with the words better, good, useful, like, advantage and best being the most frequent, 
while negative sentiment is characterised by the word problem or problems. 

RQ3: Has the impact of the Covid-19 coronavirus pandemic changed the way students view 
educational apps and platforms? 

While the analysis of the research reports from the first half of the 2019/20 academic year shows 
that the most important is content, followed by knowledge, research, method and different, the reports 
from the second half of the 2021/22 academic year the word content also ranks best, followed by good, 
experience, knowledge and better. Thus, in the post-Covid-19 reports, sentiment has come to the 
front, with 100% of these research reports expressing positive sentiment, as compared to 64% positive 
sentiment in the reports from two and a half years ago. We also observe (considering the ranking of words 
by code) that students now prioritise user experience, reason, and quality over content and personal 
preferences compared to the pre-pandemic period but are still aware of the need for exploration. Namely, 
the development of the World Wide Web and search engines has made information ‘on demand’ a reality 
(Hirsh, 2018), and technology allows them to learn at their own pace and time (Moreno, 2018). 
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Conclusions 


Our results show that the development of ICT and the related rise of educational applications 
and platforms is having a significant impact on teaching and learning. This requires teachers to take 
these applications and platforms into account and to use new teaching methods, the first prerequisite 
being a certain level of digital competence. However, we have found that this is in fact a huge burden 
for the majority of teachers, as it requires additional and continuous research and learning. It is also 
becoming increasingly apparent that traditional education must compete with e-learning opportunities, 
or the profession may gradually become meaningless for many teachers. It has also become clear that 
teachers, therefore, need to change their mindset and see their profession as adding value and building 
on what students can learn online, while at the same time becoming facilitators of such learning. 

Although most courses do not yet foresee the use of educational apps and platforms, when teachers 
use some (at least occasionally) during their teaching, it can further motivate students to participate and 
pay attention. In addition, teachers can recommend certain apps and online platforms for students to 
use for self-learning to deepen and consolidate their knowledge. Our research showed that students 
already know and use many of the apps and platforms, but it turned out that they also appeared to be 
very interested in exploring and learning about others that exist in their field of interest, as they have 
certain advantages, disadvantages, and specificities, and they were always very interested to follow the 
presentations of their classmates’ testing results. 

Most students do not actively participate during a traditional lecture (Bajpai, Biberman and Ye, 
2019), but educational apps require exactly that. Nevertheless, we would like to stress that we do not 
believe that the integration of apps and online platforms in faculty courses is always necessary or that it 
can always improve teaching and learning. However, in the spirit of the times, where students are used 
to being constantly connected to the World Wide Web and using connected devices at the same time for 
learning, searching for information, communicating, etc., we have decided to present the results of our 
survey to give an insight into students’ opinions and testing results. In addition, this encouraged students 
to think about the possibilities and options for lifelong learning that will be required of them, especially 
intensively in the future career field of the students whose research reports were the source of the data 
of our empirical research. Therefore, this assignment was proposed to final-year students in the field of 
computer science and informatics. 

The limitations of our research are clear; we only present tests of a few online platforms and 
applications, but the list can never be definitive as it is subject to constant updates. Moreover, depending 
on the fields of study, we only present the findings of a limited number of students, all from the four 
Informatics and Computer Science degree programmes from one faculty from Slovenia. Finally, although 
we do not assess the performance, usability, user experience, etc. of applications and web platforms, we 
present the main findings of students when testing them to gain insight into their thinking, our assignment 
of codes is subjective, as is the manual cleaning of the analysis results by Atlas.ti. 

Despite these limitations, the scientific contribution of our paper stems from the fact that other 
authors have not used the same methodology to shed light on the topic of the paper. This topic is also 
extremely interesting in the so-called post-coronal period, as it shows how quick the change was, due to 
the coronal period and thus has important practical implications. Moreover, the empirical results of the 
qualitative analysis of students’ opinions are particularly valuable, as their perspective is often neglected. 
At the same time, we also conducted a personal interview with each student, which allowed us to code 
and analyse the word cloud more appropriately. 

Online education is certainly an area that will continue to grow rapidly in the future. Indeed, with 
new technologies such as machine learning and 5G, we can expect further rapid growth. But here is the 
problem — teachers and students do not know what is out there already, which limits them. Therefore, 
lifelong learning also applies to teachers, who should also spend their working time exploring these 
possibilities so that they know how to use them and recommend them to students as an additional 
resource, and their role should focus even more on a mentoring style of teaching. However, this will only 
be effective if students are committed to this way of learning, which will require them to be even more 
self-controlled and self-organised. 

During the Covid-19 coronavirus pandemic teachers and students experienced a situation where 
distance teaching and learning became mandatory, requiring them to change their methods significantly, 
which further increased the use of educational applications and platforms, not to mention the various online 
communication tools. As Kodelja (2020) states, it is difficult to say how successful distance education has 
been as a result of the new coronavirus, but it seems that not all the possibilities offered by ICT have been 
exploited, due to both objective and subjective circumstances. But due to this pandemic, we cannot go 


www.ijcrsee.com 
277 


Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational 
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 
11(2), 267-279. 


back to the learning and teaching methods we used before. We need to take a further step based on the 
experiences we have gained and exploit all the possibilities offered by ICT, thoughtfully and adapted for 
different groups of students. 
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